Digital photography has gained considerable attention and popularity. This is particularly true for nonprofessional photographers. Digital photography has also found increasing use in business and commerce. Instantaneous turnaround and the simplicity with which digital images can be incorporated into electronic documents have made digital image technology one of the most desirable forms to record images.
Conventionally, many handheld devices such as digital cameras and the like have been marketed having built-in blur correcting functions for preventing an adverse effect on a captured image caused by a blur such as a camera shake caused by a user during image capture. Also, images captured on handheld devices may suffer from “motion blur” caused by unwanted shake during image capture of fast moving objects. This is particularly the case for still pictures, where the exposure time may be fairly long, as well as for light-weight handheld devices such as mobile phone cameras, which is difficult to stabilize for image capture.
Typically, various blur correcting techniques have been applied to reverse the adverse effects of motion blur on captured images. For example, US 2004/0130628 A1 describes a method where the capture of a digital image is delayed until the motion of the digital camera satisfies a particular motion criteria. According to this method, the movement factors are analyzed and the capture of the image is delayed until no further movement is detected by a tracking method and a control logic that is configured to delay capture of the image. However, once the image has been captured, no further processing or analysis of the motion is performed on the captured images.
Additionally, other blur correcting functions can be classified into two categories. First, in mechanical and optical correction systems, the motion of an image set is captured by mechanical sensors (accelerometers) and is compensated by optical means (e.g., a deformable prism) so that the image remains stable on the sensor. For example, a sensor such as a vibration gyro senses a motion and, on the basis of the result of sensing, the apical angle of a variable apical angle prism is changed or a part of an image-sensing lens is shifted, thereby preventing a blur of a captured image on an image-sensing surface.
Secondly, in digital image processing correction systems, the captured image is processed “off-line” by methods estimating, by static analysis of the image, the motion that caused the blur, then applying corrections to compensate for it. In other words, the portion of an image sensed by an image-sensing device such as a CCD, which is to be displayed on a display, is changed in accordance with a camera shake, thereby displaying an image with no blur on the display.
However, these conventional blur removal techniques described above are not often satisfactory due to a number of factors. Mechanical and optical correction systems, although they give excellent results, are used mostly on high-end devices because of the high costs associated with their integration into handheld devices. Moreover, their size makes them unattractive for fitting into ever smaller handheld devices. Another limitation is that they can only compensate for camera motion and cannot deblur parts of images in order to compensate the blurring effect of moving objects.
Similarly, digital image processing correction methods are limited in quality because the image restoration is very sensitive to an accurate model of motion. First, this motion information is not available at processing time, and the estimation of motion which can be performed from static image analysis is difficult and not robust. In particular, this system is limiting in situations where a subject motion involves a global translation of arbitrary direction, i.e., not necessarily parallel to one of the horizontal or vertical axes of the image. Moreover, it does not perform well in situations where motion is not a global translation but includes also a rotation and where motion is not homogeneous, but some areas of the image have different specific motions.
Therefore, it is desirable to implement an improved digital motion blur removal method and a corresponding device primarily used to correct blurs of motion images obtained by handheld devices such as video cameras, mobile phone cameras and the like, which avoid the above mentioned problems and can be less costly and simpler to implement.